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Abstract 

In the last 20 years, substantial investments have been made in educational technology. Not 
surprisingly, in todays Zeitgeist of educational accountability there have been increasing 
calls for empirical, research-based evidence that these investments are affecting the lives of 
teachers and students. This paper examines the ways in which teachers use technology, with 
a specific emphasis on the measurement of teachers’ technology use. Specifically, the survey 
responses of approximately 3,000 K—12 teachers are analyzed to examine the multidimen- 
sional nature of teachers’ technology use. The findings provide insight into improved strate- 
gies for conceiving of and measuring teacher technology use. (Keywords: technology, instruc- 
tion, measurement, survey research.) 


INTRODUCTION 

Over the last 20 years, substantial investments have been made in educational 
technology. Not surprisingly in today’s Zeitgeist of educational accountability 
there have been increasing calls for empirical, research-based evidence that these 
massive investments are effecting the lives of teachers and students (McNabb, 
Hawkes, & Rouk, 1999). Although there is a strong desire to examine the effect 
of technology on student learning, effects on learning must be placed in the 
context of teacher and student use. In other words, before the outcomes of 
technology integration can be studied, there must first be a clear understanding 
of how teachers and students are using technology. 

Currently, what is meant by teachers use oftechnobgy varies widely. In some cases, 
teachers’ use of technology is specific to their use while delivering instruction in the 
classroom. In other cases, teachers require students to use technology to develop 
products or to facilitate learning. In still other cases, teachers’ use includes e-mail- 
ing, lesson preparation, and record keeping, as well as personal use. Despite the 
many ways in which teachers may use technology to support their teaching, re- 
search on technology often lacks a clear definition of what is meant by teachers’ use 
oftechnobgy. In turn, a variety of approaches to measuring teachers’ use have been 
employed, most of which group together different uses into a single dimension. Al- 
though defining technology use as a unitary dimension may simplify analyses, it 
complicates efforts by researchers and school leaders to do the following: 

• provide a valid measure of technology use, 

• interpret findings about the extent to which technology is used, and 

• understand how to increase technology use. 
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In this paper, we review the several ways in which technology use has been 
measured over the past two decades and then present data that demonstrate the 
utility of employing multiple measures of teachers’ technology use. 

LITERATURE REVIEW 

The first large-scale investigation of educational technology occurred in 1986 
when Congress asked the federal OlFice of Technology Assessment (OTA) to 
compile an assessment of technology use in American schools. Through a series 
of reports (OTA, 1988, 1989, 1995), national patterns of technology integra- 
tion and use were documented. In addition, a primary finding suggested that 
the extent to which technology is used is left largely to the teacher’s discretion. 

Ten years later. Congress requested OTA “to revisit the issue of teachers and technol- 
ogy in K-12 schools in depth” (OTA, 1995). In a 1995 OTA report, the authors noted 
that previous research on teachers’ use of technology employed different definitions of 
what constimted technology use. In turn, these different definitions led to confusing and 
sometimes contradiaory findings regarding teachers’ use of technology. For example, a 
1 992 International Association for the Evaluation of Educational Achievement (lEA) 
survey defined a “computer-using teacher” as someone who “sometimes” used computers 
with students. A year later, Becker (1994) employed a more explicit definition of a com- 
puter-using teacher, for which at least 90% of the teachers’ students were required to use 
a computer in their class in some way during the year. Thus, the lEA defined use of 
technology in terms of the teachers’ use for instructional delivery while Becker de- 
fined use in terms of the students’ use of technology during class time. Not surpris- 
ingly, these two different definitions of a “computer-using teacher” yielded different 
impressions of the technology use. In 1992, the lEA study classified 75% of U.S. 
teachers as “computer-using teachers,” while Becker’s criteria yielded about one 
third of that (approximately 25%) (OTA, 1995). This confusion and inconsistency 
led the OTA to remark: “Thus, the percentage of teachers classified as computer- 
using teachers is quite variable and becomes smaller as definitions of use become 
more stringent” (p. 103). 

During the mid 1990s, several advances in computer-based technologies came 
together and allowed teachers to use technology to support their teaching in an in- 
creasing variety of ways. Whereas instructional uses of computers had been limited 
largely to word processing and computer programming, teachers were now able to 
perform multimedia presentations and computer-based simulations. With the in- 
troduction of the Internet into the classroom, teachers were also able to incorporate 
activities that tapped the World Wide Web. Outside of class time, software for 
record keeping and test development provided teachers with new ways of using 
computers to support their teaching. In addition, the Internet allowed teachers ac- 
cess to additional resources when planning lessons (Becker, 1999) and allowed 
teachers to use e-mail to communicate with their colleagues, administrative leaders, 
students, and parents (Lerman, 1998). Naturally, as the variety of ways in which 
teachers could use technology increased, defining a technology-using teacher be- 
came more complicated. (For a complete review of the timeline of changes in the 
ways teachers use computers, see Becker’s Analysis and Trends of School Use of New 
Information Technologies, 1994.) 
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QUANTIFYING TEACHERS’ MODERN TECHNOLOGY USE (1994-2002) 

Since 1994, the National Center for Educational Statistics (NCES) has con- 
ducted a series of surveys on public school teachers’ access to and use of com- 
puters and the Internet. In a 2000 report, the NCES differentiated among types 
of teachers’ technology uses and reported that, although the vast majority of 
teachers were using technology for some aspects of their professional activities, 
non-instructive technology uses were pervasive. For example, the NCES re- 
ported that: 

• 85% of teachers use a computer to create instructional materials at home and 
78% do so at school. 

• approximately half of all teachers use computers for administrative record 
keeping at school and at home. 

• approximately half of all teachers use e-mail to “communicate with col- 
leagues” and about a quarter of teachers communicate with parents by e-mail. 

• approximately 20% of teachers post homework and assignments on the 
Internet. 

Recognizing instructional use as a separate facet of technology use, the NCES 
also reported that 53% of all public school teachers who have a computer at 
school are using it for instruction during regular class time. In a summary re- 
port of the 2000 NCES document, Rowand (2000) articulated several facets of 
teacher technology usage that were measured by the NCES survey. These facets 
included the teacher using technology to/for the following (in descending order 
of frequency of use): 

• creating instructional materials 

• keeping administrative records 

• communicating with colleagues 

• gathering information for planning lessons 

• presenting multimedia classroom presentations 

• accessing research and best practices for teaching 

• communicating with parents or students, and 

• accessing model lesson plans 

Besides the work by the NCES, perhaps the largest recent study of teachers’ 
technology practices was the Teaching, Learning, and Computing (TEC) sur- 
vey that was conducted in 1 998 and has generated nine full reports, which de- 
tail the practices and beliefs of a representative sample of United States teachers 
(Becker, 1999; Ravitz, Wong, & Becker, 1998, 1999, 2000; Ravitz & Wong, 
2000). Like the NCES, Becker and his colleagues documented that teachers’ 
and students’ use of computers was becoming more varied and more wide- 
spread. For example, 71% of Grade 4-12 teachers reported that they had their 
students use a computer at least once in some way during the 1997-1998 
school year. Of those teachers who reported that they do not use technology 
with their students, three-fourths reported that they do use technology them- 
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selves for non-instructional purposes. In fact, the most frequent uses of tech- 
nology across all subject areas was not instructional use but “professional uses 
of technology related to their day-to-day needs” (Becker, 1999, p. 31). One of 
the most frequent uses of technology reported by teachers was making hand- 
outs for class (66% of all teachers reported making handouts at least weekly). 

In addition, almost half of the teachers reported using a computer at least 
weekly for record keeping and student grading, two-thirds reported using the 
Internet for lesson planning, and 68% reported using e-mail for communica- 
tion. In short, the 1998 TLC survey data indicated that the majority of teach- 
ers were using technology to support their teaching, but much of this use oc- 
curred outside of class time. 

This finding was echoed by Cuban (2001), whose book Oversold and 
Underused has led many to question the impact of limited numbers of comput- 
ers in classrooms on teaching and learning. It is important to note that when 
making the connection between technology use and teaching, Cuban separated 
technology use during class time and out of class time. Despite repeated efforts 
to distinguish between uses of technology during and outside of class time, 
when making the argument that computers are underused as instructional tools, 
Cuban employs a definition of technology use that is exclusive of technology 
for communication, lesson planning and preparation, grading, and record keep- 
ing. In other words, Cuban addresses the impacts (or lack thereof) of technol- 
ogy on instructional practices using a less than complete measure of what con- 
stitutes teachers’ technology use. 

It is clear, both in theoretical and investigative research, that defining and 
measuring teachers’ use of technology has increased in complexity as technol- 
ogy has become more advanced, varied, and pervasive in the educational sys- 
tem. In actuality, very little has changed since the mid 1990s, when the U.S. 
Department of Education raised concern about the different ways in which 
technology use was being defined and measured. Today, several researchers and 
organizations have developed their own definitions and measures of technology 
use to examine the extent of technology use and to assess the impact of tech- 
nology use on teaching and learning. Without question, instruments such as 
those developed by the CEO Forum and the International Society for Technol- 
ogy in Education (ISTE) appear to be effective in spurring reflection among 
school leaders and discussion regarding technology’s impact in schools. Fre- 
quently these instruments collect information on a variety of different types of 
teachers’ technology use and then collapse the data into a single generic “tech- 
nology use” variable. Unfortunately, the amalgamated measure may be inad- 
equate both for understanding the extent to which technology is being used by 
teachers and for assessing the impact of technology on learning outcomes. 
Moreover, there is a strong likelihood that the school leaders who rely upon 
this information for decision-making will interpret findings in a number of 
different ways. For example, some may interpret one measure of teachers’ tech- 
nology use solely as teachers’ use of technology for delivery, while others may 
view it as a generic measure of the collected technology skills and uses of a 
teacher. 
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Recognizing the importance of how technology use is both defined and mea- 
sured, the remainder of this paper uses data collected as part of the Use, Sup- 
port, and Effect of Instructional Technology (USEIT) Study to describe efforts 
to develop multiple measures of teachers’ technology use and to provide ex- 
amples of how a multifaceted approach to measuring teachers’ technology use 
holds the potential to provide deeper insight into how technology use varies 
across settings. Based on the findings described here, implications for future 
definitions and measurement of technology use are discussed. Throughout the 
present work, the term technology refers specifically to computer-based tech- 
nologies and includes personal computers, LCD projectors, and Palm Pilots. 
Prior to examining technology use, we provide a brief overview of the USEIT 
study sample and design. 

SAMPLE AND METHODOLOGY 

To explore the utility of a multi-dimensional definition of technology use, the 
analysis uses data collected as part of the USEIT Study. Working with 22 school 
districts located throughout Massachusetts, the USEIT Study examines the use 
of educational technologies by teachers and students, the factors that influence 
these uses, and the effect of these uses on student learning. The 3-year study be- 
gan during the spring of 2001 and included survey responses from 2,894 K-12 
mathematics. English/language arts, science, social studies, and elementary 
school classroom teachers from schools across 22 districts. As described in 
greater detail by Russell, Bebell, and O’Dwyer (2003), the sample includes a 
broad spectrum of teachers across grade levels and school types (elementary, 
middle, and high school), with each grade level represented by at least 230 
teachers. The sample also includes at least 470 teachers from each of the four 
main subject areas as well as 1,279 self-contained elementary school classroom 
teachers who reported teaching multiple subject areas. 

The USEIT teacher survey was developed based on current literature, was de- 
signed to focus on a broad range of issues related to teacher and student use of 
technology, and included 45 items that focused on uses of technology both in 
and outside of the classroom by both teachers and their students. Twelve of 
these items asked teachers about the frequency with which they used a specific 
piece of hardware, such as a desktop computer in their classroom, shared 
laptops, an LCD projector, or a scanner. Because these survey items did not 
specify whether it was the teachers or their students using the devices, they are 
excluded from the analyses presented below. Thus, these analyses began by fo- 
cusing on the remaining 33 items, 2 1 of which were ultimately used to form 
seven separate scales of technology use. 

DEEINING TECHNOLOGY USE AS MULTI DIMENSIONAL 

Two approaches are often used to represent teacher technology use: (a) a com- 
posite measure that represents an index of general technology use is calculated, 
or (b) multiple measures, each of which represents a specific category or type of 
technology use, are generated. The first approach creates a single aggregate mea- 
sure that represents the frequency with which a teacher uses technology for a va- 
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riety of purposes. Because an increased value in a teacher’s response to each sur- 
vey item corresponds to more frequent use, the items are summed to generate a 
composite measure that represents the frequency of technology use. 

A histogram of the summed composite measure of technology use for the sample 
of 2,628 teachers is displayed in Figure 1. As seen in Figure 1, the composite tech- 
nology use measure is approximately normally distributed (skewness = -.04) with 
the majority of respondents clustered in the middle of the distribution and with a 
fairly equal number of respondents at the respective ends of the distribution. 
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Figure 1: Single composite measurement of teacher technology use 

The second approach to measuring teacher technology use involves examin- 
ing the specific ways in which teachers make use of technology. In this case, 
multiple measures (i.e., scales) for the specific ways that teachers use technology 
are constructed from related survey items. As documented in Russell, O’Dwyer, 
Bebell, & Miranda (2003), principal component analyses were used to develop 
seven separate scales that measure teachers’ technology use. These seven scales 
are as follows: 

• Teachers’ use of technology for class preparation (Preparation) 

• Teachers’ professional e-mail use (Professional E-mail) 

• Teachers’ use of technology for delivering instruction (Delivering Instruction) 

• Teachers’ use of technology for accommodation (Accommodation) 

• Teacher-directed student use of technology during class time (Student Use) 

• Teacher-directed student use of technology to create products (Student Products) 

• Teachers’ use of technology for grading (Grading) 



Std. Dev = 
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Figure 2: Distribution and mean frequency of use for items comprising the seven 
categories of teacher technology use 

These seven categories of teacher technology use are displayed in Figure 2 
along with the distribution and mean response for each of the items used to 
form each of the seven scales. 


Journal of Research on Technology in Education 5 1 

Copyright © 2004, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), isre@iste.org, www.iste.org. All rights reserved. 



As seen in Figure 2, the number of items used to form each category of use 
ranges from one to five. Also note that the distribution of responses and mean 
response varies considerably across the individual items. For example, the distri- 
bution of responses for the item that asks teachers how often they make hand- 
outs for students using computers is negatively skewed, with the vast majority 
of teachers reporting that they do this several times a week or several times a 
month. For this item, the mean response indicates that, on average, teachers use 
computers often to make handouts. In contrast, the distribution of responses 
for the item that asks teachers how often they have students perform research 
using the Internet or CD-ROMs during class time has a relatively normal distri- 
bution with a mean that is just below the mid-point of the scale. In further con- 
trast, the item that asks teachers how often they ask students to produce multi- 
media projects has a large positive skew, with most teachers responding that 
they never have students create these type of projects. 

Although examining teacher responses at the item level is informative and 
may reveal interesting patterns across items, patterns become easier to identify 
when items that focus on related uses of technology are combined into a single 
measure. As described above, principal component analyses were used to iden- 
tify the items that have strong inter-correlations and thus can be combined to 
form a single measure that represents a specific category of technology use. Fur- 
thermore, because the same five-point response options, which correspond to 
the frequency of technology use, were used for all of the items that form the 
seven categories of technology use, an aggregate score for each category of use 
was calculated by summing each teacher’s response across the survey items 
forming each category and then dividing by the number of items related to that 
category. The aggregate scores for each category of technology use are displayed 
in Figure 3. 

As shown in Figure 3, teachers reported that the extent of technology use is 
highest for preparation. The next most frequent use is for e-mail, followed by 
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Figure 3: Degree of use for seven categories of technology use 
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teacher-directed student use of technology and the use of technology for grad- 
ing. Note that with the exception of preparation, teachers (on average) report 
low to moderate levels of use for each category, with use for accommodation 
and for the creation of student products occurring least frequently. It is impor- 
tant to note, however, that the sample of teachers upon which these analyses are 
based excludes special education teachers who are perhaps more likely to de- 
velop accommodations for lessons. 

Although it is procedurally easier to form a single composite score by combin- 
ing responses across all items than it is to form multiple categories of use, a 
richer understanding of how technology is and is not being used by teachers re- 
sults when multiple categories of use are employed. As an example, recall that 
the USEIT sample of teachers was normally distributed on the generic measure 
of technology use (Figure 1). This normal distribution indicates that most 
teachers are making moderate use of technology and that relatively few teachers 
are using technology heavily or not at all. The distribution of responses for the 
seven separate technology measures, however, suggest that the distribution of 
use varies dramatically across the separate categories of use. As shown in Figure 
4, the distribution of teacher use of technology for instruction is positively 
skewed (skewness = 1.09) rather than normally distributed. This indicates that 
the majority of the teachers in this sample do not use technology for instruction 
very often. In fact, nearly 900 teachers indicated that they never use technology 
to deliver instruction. 
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Figure 4: Histogram of teachers’ use of technology for Delivering Instruction 

In contrast. Figure 5 indicates that the use of technology for preparation is 
negatively skewed (skewness = -1.12), with most of the teachers reporting high 
frequency of preparation use. 

Figure 6 displays the histograms for the five remaining technology use mea- 
sures. As with Instructional Use, the distributions for Student Product use 
(1.15) and Accommodation (1.04) have large positive skews. Grading Use 
(0.60) also has a weak positive skew, while Teacher-Directed Student Use dur- 
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ing class time (0.1 1) is relatively normally distributed. Professional E-mail Use 
(skewness =-0.18), however, appears bi-modal, with a large percentage of teach- 
ers reporting frequent use and a large portion of the sample reporting no use. 
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Figure 5: Histogram of teachers’ use of technology for Preparation 
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Figure 6: Histogram of the five remaining measures of teachers’ technology use 

In short, by measuring teachers’ use of technology using specific measures, 
we see important differences in the frequency with which teachers use technol- 
ogy for a specific use as well as differences in how these uses vary across teach- 
ers. When compared to a single generic measure of technology use, multiple 
measures of specific technology use offer a more nuanced understanding of 
how teachers are using technology and how these uses vary among teachers. 


CORRELATION AMONG TECHNOLOGY USES 

By developing separate measures of teachers’ technology use, we are not in- 
ferring that each individual measure is unrelated to the other technology use 
measures. Indeed, it would be reasonable to assume that all of the measures 
have some degree of relation to each other. The strength of the relationships 
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among the seven technology uses are examined via Pearson correlation coeffi- 
cients, which are presented in Table 1 . 


Table 1: Correlation Table of tbe Seven Specific Teacher Technology 

Measures 
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0.11 

0.17 

0.15 

0.24 
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Table 1 shows that the correlations among the seven teacher technology use 
measures are all positive, but generally indicate weak to moderate relationships. 
The positive inter-correlations suggest that teachers who use technology for one 
purpose are, on average, likely to use technology for other purposes. Likewise, a 
teacher who never uses one form of technology is likely to be an infrequent user 
of other technologies. However, the moderate to weak correlations also suggest 
that there is considerable variation between the extent to which teachers use 
technology for one purpose and the extent to which they use technology for an- 
other purpose. 

Across the seven categories of technology use, the median correlation is 0.26. 
When examining the correlations between any two of the technology uses, two 
measures have inter-correlation coefficients that are larger than 0.4 (Delivery 
correlated with Student Use and Student Use correlated with Student Prod- 
ucts). Aside from these relationships, there are four examples of correlations 
above 0.3 (Accommodation correlated with Student Use, Delivery correlated 
with Student Products, Preparation correlated with Professional E-mail, and 
Preparation correlated with Student Products). Again, it is logical that there is a 
positive relationship between these pairs of measures. Yet, the relatively weak to 
moderate correlations among each of the uses suggest that each teacher technol- 
ogy category does represent a separate aspect of technology use. 

HOW MULTIPLE MEASURES OE TECHNOLOGY USE IMPROVE 
UNDERSTANDING 

Although the seven teacher technology use measures are weakly to moderately 
related to each other, the analyses presented above provide evidence that (a) 
each measure does represent a separate and distinct category of technology use, 
and (b) the frequency and distribution of technology use varies considerably 
across the seven measures. In this section, we examine how the use of separate 
measures of technology use provides insight into the ways in which technology 
use varies across different groups of teachers. Specifically, we examine patterns 
of use for (a) teachers who have been in the profession for different lengths of 
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time, (b) teachers who teach in different school types (i.e., elementary versus 
middle/high school), and (c) teachers who teach different subject areas (En- 
glish/language arts, mathematics, social studies/geography, and science). These 
analyses are presented to illustrate how our understanding of technology use 
changes when a single generic measure of use versus multiple specific measures 
are used. 

TECHNOLOGY USE BY YEARS TEACHING 

It is commonly believed that as new teachers — who have grown up with and 
are comfortable using technology — enter the teaching profession, technology 
use in schools will increase (National Center for Education Statistics, 2000). 
When examining use of technology by teachers using a generic measure that 
comprises a variety of types of technology use, it appears that the frequency 
with which teachers use technology does not vary noticeably across the number 
of years in the profession. As seen in Figure 7, teachers who are brand new to 
the profession report almost the same amount of use as do teachers who have 
been in the profession for 1 1 or more years. 



Number of Years Teacbing 

Figure 7: Comparison of generic technology use measure across the number of 
years teacher has taught 

However, when multiple measures of technology use are employed, the pat- 
tern changes noticeably. As depicted in Figure 8, newer teachers report higher 
levels of technology use for preparation and slightly higher levels of use for ac- 
commodation than do more experienced teachers. Conversely, new teachers re- 
port less frequent use of technology for delivery and report asking students to 
use technology during class time less frequently than do their more experienced 
colleagues. 
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Number of Years Teaching 

Figure 8: Comparison of multiple technology measures across the number of 
years teacher has taught. 

A similar pattern occurs when examining the relationship between technology 
use and school type. As shown in Figure 9, the frequency with which teachers 
report using technology appears to be nearly identical in elementary and in 
middle/high schools based on a generic measure of technology use. However, 
interesting differences appear when multiple measures of technology use are ex- 
amined. Although teachers in both settings report about the same amount of 
use for Delivering Instruction, for Professional E-mail, and for Student Prod- 
ucts, elementary teachers report using technology to accommodate lessons and 
asking their students to use technology during class time more frequently than 
do the middle/high school teachers. Conversely, the middle/high school teach- 
ers report using technology for preparation at a higher frequency than do the el- 
ementary school teachers. By far, however, the largest difference in use occurs 
for grading, with middle/high school teachers report occurring much more fre- 
quently than their elementary school counterparts. 

Although the differences are less dramatic across subject areas, the way in which 
technology use is defined also influences our perception of how technology use dif- 
fers across subject areas. As seen in Figure 1 0, there appears to be little difference in 
the frequency of technology use across English/language arts, social studies/geogra- 
phy and science teachers when technology use is defined with a generic measure. 
Using this generic measure, it also appears that mathematics teachers use technol- 
ogy less frequently than the other three groups of teachers. When employing mul- 
tiple measures of technology use, we see a similar pattern for most uses, with math- 
ematics teachers reporting the lowest levels of use for Preparation, Student Use 
during class time, and Student Products. For each of these uses, however, the differ- 
ence in use between mathematics teachers and teachers of the other subjects is 
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larger than it is for the generic use measure. However, mathematics teachers appear 
to use technology to deliver instruction about as frequently as any other group and 
report noticeably higher levels of use for grading. Although less dramatic, the way 
in which technology use differs between the other subject areas also changes when 
individual categories of use are examined. Perhaps most notably, whereas English, 
social studies, and science teachers appear similar with respect to the generic mea- 
sure, English teachers report using technology for grading less and use technology 
for accommodations more than teachers of the other two subject areas. 



Figure 9: Comparison of different technology measures across school type 
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Figure 10: Comparison of different technology measures across school type 
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Although there are many possible reasons that may explain the differences in 
use noted above, our purpose here for examining patterns of use is to demon- 
strate how these patterns differ depending upon how one defines and measures 
technology use. Whereas there appears to be little difference in the frequency 
with which teachers use technology based on their years teaching, their school 
type, or across most subject areas (except mathematics) when a generic measure 
of technology use is employed, important differences appear when technology 
use is examined as a multi-dimensional construct. 

DISCUSSION 

Investments in educational technology have sparked important questions 
about the impact of technology on teaching and learning. In turn, leaders and 
researchers at the district, state, and national levels are making efforts to both 
increase use of technology by teachers and students and to examine technology’s 
effects on student learning. In many cases, however, definitions of technology 
use vary substantially across settings and studies, and technology use is often ex- 
amined in a generic fashion. The analyses presented here demonstrate the value 
of conceiving of technology use as multiple categories or types of use rather 
than a single generic construct. Using 2 1 survey items that focus on specific 
uses of technology, the analyses presented above demonstrate the following: 

• Separate measures that represent distinct categories of technology use can be 
formed 

• Although these measures are correlated positively with each other, the 
strength of the relationships are weak enough to suggest that each category 
represents a separate and distinct type of use 

• The use of distinct measures versus a generic measure provides a richer, more 
nuanced understanding of how technology use differs across factors such as 
teacher tenure, school type, and subject area taught. 

The implications of this approach are especially applicable to the future devel- 
opment of surveys and other research instruments designed to measure teachers’ 
use of technology. For example, a district interested in documenting the extent 
to which teachers are using technology or the extent to which teachers’ use of 
technology changes in response to the acquisition of more resources or the pro- 
vision of professional development are likely to develop a richer understanding 
by collecting information about the specific types of teachers’ technology use 
rather than simply measuring its generic presence or absence. 

Using a multi-faceted approach to measure teachers’ use of technology also 
brings to bear the general issue of how complicated and varied technology use 
actually is in today’s schools. In good faith, a principal can no longer evaluate a 
teacher based on whether the teacher is using technology or not, but rather the 
question should evolve to include how a teacher is making use of various tech- 
nologies and for what purposes. 

In fact, in keeping with previous research findings, our examination of educa- 
tional technology use shows that the majority of teachers’ use of technology 
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goes on behind the scenes with lesson preparation, grading, and professional e- 
mail use rather than instructional use or teacher-directed student use (Cuban, 
2001; Becker, 1999; Russell, Bebell, O’Dwyer, & O’Connor, 2003). For this 
reason, the traditional methodological tool of classroom observations would fail 
to capture these activities in an evaluation or research study. Similarly, studies 
that focus entirely on student-reported data (Edwards, 2002) would also fail to 
capture the most frequent and pervasive teacher uses of technology. 

This point is also directly relevant when examining the relationship between 
technology use and its impacts on student learning. Although several studies 
have documented positive effects of technology use on student learning when 
the technology is used directly by students (see Goldberg, Russell, & Cook, 
2003; Kulik, 1994; and Sivin-Kachala, 1998 for meta-analyses of nearly 700 
studies), the analyses presented above suggest that teacher-directed student use 
of technology during class time represents just one category of use. Moreover, 
teacher-directed student use during class time is reported to occur less fre- 
quently in comparison to teachers’ use for preparation or communication. Al- 
though it is unquestionably important to understand how student use of tech- 
nology impacts student learning, it is equally important to examine the 
relationship between student learning and other uses of technology by teachers 
that directly support instruction. Clearly, this type of analysis can only be con- 
ducted if multiple measures of technology use are employed. 

Finally, it is important to recognize that although we strongly advocate for 
multiple measures of technology use, we are not suggesting that the uses we 
have employed represent a definitive body of uses or that the items used to form 
each measure are exhaustive. To the contrary, we believe that as researchers and 
educators who are familiar with educational technology consider the full range 
of ways in which technology is currently being used, additional categories of use 
will be identified. Similarly, as new technologies become available and as ubiq- 
uitous computing becomes more prominent in schools, specific uses of technol- 
ogy will emerge and categories of technology use will expand. Although it may 
seem efficient to “borrow” surveys or items that have been used for other re- 
search or evaluations, doing so may fail to capture the full range in which teach- 
ers are using existing and recently acquired technologies for a variety of pur- 
poses. It is for these reasons that we encourage schools, districts, and researchers 
who will be using surveys to document technology use, to begin by defining the 
categories or types of use of interest and to then develop items related to each 
category of use. 

As an example, when developing the teacher survey for the USEIT Study, we 
theorized that teachers’ technology use fell into four categories and developed 
several items related to each category. These theoretical categories included 
teachers’ professional technology use outside of class time, teachers’ use of tech- 
nology during class time (including student-directed uses), teachers assigning 
work that required students to use specific technology, and teachers’ communi- 
cation through e-mail. As described above, our analyses of the teachers’ re- 
sponses to the survey led us to identify seven specific and independent uses of 
technology by teachers. This expansion occurred because three uses, which 
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originally were subsumed within another category of use, were found to be in- 
dependent. As an example, we had originally believed that grading was part of 
professional use outside of the classroom, and thus would be strongly related to 
creating tests, handouts, and using the Internet to prepare for lessons. Similarly, 
we believed that teachers’ use of technology to deliver instruction and to make 
accommodations were components of technology use during class time. How- 
ever, our analyses indicate that teachers’ technology use for delivery and to ac- 
commodate lessons operated independently of teachers asking students to use 
technology during class time and that teachers’ use of technology for grading 
operated independently of preparation. Having learned this, before conducting 
similar research in the future, we could increase the reliability of our measures 
and the amount of information provided about each category of use by devel- 
oping additional items to measure the categories of use that have emerged from 
our analyses of the USEIT Survey data. 

In closing, during the past decades a wide variety of computer-based tech- 
nologies that can and are being used for educational purposes have emerged. 
Without question, the variety of technologies and the multiple ways in which 
some technologies can be used for educational purposes complicates efforts to 
document technology use and the effect of these uses on teaching and learning. 
As we have shown using a limited number of survey items, simply conceiving of 
a variety of uses of technology as a single generic measure of technology use 
masks far more than it reveals. 
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